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ABSTRACT 

Large  scale  immersive  visualization  environments 
developed  for  Future  Force  Warrior  (FFW)  and  Future 
Combat  Systems  (FCS)  will  dramatically  increase  the 
volume  of  data  available  to  commanders.  At  the  same 
time,  immersive  three  dimensional  (3D)  visualization 
stereoscopic  technologies  can  improve  the  commander’s 
ability  to  comprehend  this  avalanche  of  information  and 
act  upon  it  quickly  and  decisively.  Advanced 
visualization  and  interaction  technologies  will  also 
support  the  commanders’  need  for  mission 
planning/rehearsal,  situational  awareness  and  after  action 
reviews. 

The  U.S.  Army  Communications  Electronics 
Research  Development  &  Engineering  Command 
(CERDEC)  Command  &  Control  Directorate  (C2D)  has 
developed  several  immersive  C2  applications  to  support 
the  future  commander’s  ability  to  make  decisions  faster 
and  more  accurately.  Utilizing  a  Fakespace  stereoscopic 
three-dimensional  (3D)  C4ISR  (Command,  Control, 
Communications,  Computers,  Intelligence,  Surveillance 
and  Reconnaissance)  Automated  Virtual  Environment 
(CAVE)  Laboratory  and  a  Fakespace  Rapidly  Operational 
Virtual  Reality  (ROVR™)  system,  C2D  has  done 
extensive  research  in  its  lab  located  at  Fort  Monmouth,  NJ 
and  in  the  field  at  the  C4ISR  On-The-Move  (OTM)  2006 
Experiment  conducted  at  Fort  Dix,  NJ. 

1.  INTRODUCTION 

The  CERDEC  C2D  CAVE  Lab  was  created  in  2004. 
Since  that  time  the  team,  which  consists  of  government 
and  contractor  personnel,  has  been  conducting  research 
and  development  in  the  application  of  immersive 
technologies  and  advanced  visualization  techniques  to 
assist  commanders  in  their  decision-making  processes. 


highly  mobile,  ruggedized  ROVR™  system,  see  Figure  2, 
which  can  be  scaled  to  meet  specific  user  needs.  The 
team  has  also  developed  software,  the  Immersive 
Interactive  Command  &  Control  Environment  (I2C2E), 
and  the  C2  Two-Handed  Interface  (C2THI)  for  the 
presentation  and  manipulation  of  stereoscopic  3D  scenes. 
The  team  has  also  developed  a  tactical  message  parser 
and  database  that  allows  for  the  reception  and  storage  of 
tactical  data,  and  an  interface  to  the  I2C2E  for  the  display 
of  that  data  in  the  3D  environment. 


Figure  1.  The  CAVE® 


2.  THE  C2D  ROVR  SYSTEM 


To  date,  the  CAVE  team  has  built,  using 
commercial-off-the-shelf  (COTS)  equipment,  a  room¬ 
sized  immersive  environment,  see  Figure  1,  and  the 
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A  single  ROVR  consists  of  a  large  6’  x  7.5’ 
projection  screen,  a  high-bright  three  chip  digital  light 
processing  (DLP™)  projector  and  a  computer  that 
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contains  the  system  software  necessary  to  drive  the 
projectors.  The  ROVR™  supports  active  or  passive 
stereoscopic,  or  monoscopic  viewing,  and  can  support 
rear  or  front  projection. 

The  C2D  ROVR™  consists  of  three  edge-aligned, 
rear  projection  ROVR™  systems.  It  includes  three  2,000 
lumen  (brightness)  3 -chip  DLP™  projectors  and  a  four 
computer  cluster  that  includes  three  nodes  to  drive  the 
projectors  and  a  master  computer  to  coordinate  and 
control  the  nodes.  For  user  navigation  and  orientation  in 
the  scene,  the  system  uses  an  InterSense  IS900  inertial 
acoustic  motion  tracking  system  with  a  six  degree-of- 
freedom  (6DOF)  tracked  wand. 

The  ROVR™  is  highly  mobile  and  ruggedized.  Each 
screen  and  frame  fits  neatly  into  a  duffle  bag-like  carry 
case  and  each  projector  is  mounted  in  an  Air  Transport 
Association  (ATA)  category  1  case.  The  ROVR™ 
system  is  easily  transportable  in  most  truck- style  vehicles. 
When  fully  assembled  and  setup  in  the  arena  layout  the 
C2D  ROVR™  system’s  footprint  is  approximately  6.5’  x 
26’ x  9’. 


3.  THE  C2D  APPLICATIONS 

C2D’s  application  software  permits  the  user  to 
interact  with  the  simulated  environment.  The  basic 
functionalities  include  the  terrain  models  that  provide  the 
3D  simulation  of  the  modeled  area;  a  user- interface  that 
permits  selection  of  the  operating  mode,  live  or  playback; 
the  Event  Data  Manager  which  receives,  stores  and 
forwards  tactical  messages;  and  the  presentation  software 
which  displays  the  scenes  and  permits  the  user  to  interact 
with  the  immersive  environment. 

3.1  Terrain  Models 

The  focus  of  the  terrain  model  development  was  on 
several  training  ranges  located  at  Fort  Dix,  NJ  where 
initial  field  testing  of  the  ROVR™  system  and  the 
application  software  was  performed  during  the  C4ISR 
OTM  2006  Experiment.  Selection  of  the  specific  ranges 
was  coordinated  with  the  Product  Manager  (PM)  C4ISR 
OTM  who  conducts  the  annual  experiments  at  Fort  Dix. 

For  this  effort  data  collection  was  performed  using  a 
manual  process.  This  consisted  of  visiting  each  range 
with  a  global  positioning  system  (GPS),  a  laser-based 
distance  measuring  device  and  a  digital  camera.  At  each 
range  the  location  of  each  structure  was  documented 
using  the  GPS.  A  structure  is  defined  as  all  permanent 
features  of  the  range  including  buildings,  bunkers, 
vehicles,  etc.  The  dimensions  of  each  structure  were  also 
measured  and  photos  of  each  structure  were  taken  from 
various  angles  and  distances. 


Utilizing  the  collected  data,  a  3D  modeler 
constructed  high-fidelity  models  of  each  range  using  the 
Autodesk  3DS  Max  and  Newtek  Lightwave  modeling 
software  packages,  see  Figure  3.  Using  terrain  data  of  the 
Fort  Dix,  McGuire  Air  Force  Base  and  Lakehurst  Naval 
Air  Station  area  from  the  United  States  Geological  Survey 
(USGS)  the  range  models  were  precisely  geo-located  on 
the  USGS  map.  This  allows  users  to  navigate  from  range 
to  range  with  no  discontinuities  in  terrain.  The  entire 
modeling  effort  for  the  ranges  utilized  in  the  C4ISR  OTM 
2006  Experiment  took  approximately  four  months  for  a 
single  3D  modeler. 


Figure  3.  Tower  model 


3.2  The  User-Interface 

The  user- interface  (UI)  has  two  primary  modes: 
Live  Mode  and  Review  Mode.  Live  Mode  displays 
tactical  messages  and  unit  positions  in  real  time.  Review 
Mode  provides  the  operator  with  the  ability  to  view  past 
events  and  positions.  The  operator  can  toggle  between 
these  two  modes  depending  on  which  one  supports  the 
task  they  are  trying  to  accomplish. 

In  both  modes,  the  UI  presents  the  operator  with  a 
DVR-like  (digital  video  recorder)  interface.  The  interface 
allows  the  operator  to  move  to  and  begin  playback  from 
any  point  in  the  specified  mission.  The  UI  also  has  fast- 
forward,  rewind  and  pause  features  as  well  as  the  ability 
to  adjust  playback  speed  from  lx  to  64x.  When  the  UI  is 
in  Live  Mode,  these  controls  allow  the  operator  to 
temporarily  back  up  to  any  point  in  the  mission  and  then 
quickly  return  to  live  data. 

In  order  to  decrease  visual  clutter,  the  UI  allows  the 
operator  to  filter  out  entities  individually  or  as  a  collection 
specified  by  type  (e.g.  UAV,  UGS),  name  or  reporting 
hierarchy. 

While  the  C2D  application  allows  operators  to  freely 
navigate  throughout  the  terrain,  it  is  often  useful  to 
quickly  jump  to  a  pre-defmed  location.  The  UI  allows  the 
user  to  store  named  locations  for  instant  navigation  at  a 
later  time. 
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3.3  The  Event  Data  Manager 

The  Event  Data  Manager  (EDM)  provides  the 
mechanism  for  interfacing  the  system  to  the  tactical 
network,  receiving  and  parsing  tactical  messages,  storing 
tactical  data  and  injecting  tactical  data  into  the  system  for 
display.  Currently  the  EDM  provides  receive-only 
capability. 

Development  of  the  EDM  was  coordinated  through 
the  PM  C4ISR  OTM  to  ensure  it  implemented  the  proper 
messaging  standard  for  the  2006  experiment.  The 
implemented  messaging  standard  is  the  variable  message 
format  (VMF)  series  of  messages.  VMF  is  a  bit-oriented 
tactical  message  standard  which  specifies  how  tactical 
data  is  transmitted,  received  and  displayed.  The  types  of 
messages  received  were  Blue  Force  (BluFor)  position 
reports  and  Opposing  Force  (OpFor)  spot  (entity  data) 
reports. 

After  parsing  a  VMF  message,  the  tactical  data 
contained  within  is  stored  in  the  EDM  database.  If  the 
system  is  operating  in  live  mode,  the  tactical  data  is 
injected  directly  to  the  application  software  for  immediate 
display. 

In  addition  to  storing  tactical  data,  the  database 
contains  additional  information  which  describes  the 
mapping  of  an  entity  to  its  specific  Standard  Warfighter 
Symbology  (MIF-STD-2525)  symbol  and  physical 
representation,  e.g.  soldier,  ground  vehicle,  air  vehicle, 
etc.  icon. 

3.4  The  Presentation  Software 

The  presentation  software  fuses  the  terrain,  structure 
and  entity  models  with  input  from  the  UI  and  EDM  to 
complete  the  immersive  environment.  User  input  from 
the  6DOF  wand,  tracking  system  and  UI  allows 
navigation  within  and  control  of  the  3D  scene. 

The  presentation  software  has  been  designed  to  take 
advantage  of  the  unique  features  of  a  large  scale,  fully 
immersive  and  interactive  environment.  For  venues 
which  offer  limited  options  for  a  display  system,  the 
presentation  software  is  capable  of  running  on  a  single 
laptop  or  desktop  computer. 

The  scalability  of  the  solution  is  critical  as  it  allows 
users  in  disparate  locations  with  systems  of  varying 
capabilities  to  use  the  same,  familiar  application  and  UI. 

4.  C4ISR  OTM  EXPERIMENT 

The  CERDEC  C4ISR  OTM  Experiment  is 
conducted  annually  in  an  operationally  relevant 


environment  using  live,  virtual  and  constructive  scenarios. 
The  experiment  is  designed  to  identify  and  mitigate  risks 
to  current  force  technology  insertions  and  FCS  spinouts 
early  in  the  development  cycle. 

The  C4ISR  OTM  2006  Experiment  was  conducted 
from  10-17  July  and  consisted  of  a  representation  of  a 
FCS  reconnaissance  troop  supported  by  a  squad-sized 
infantry  quick  reaction  force.  This  Blue  Force  consisted 
of  soldiers  from  the  2nd  Battalion,  113th  Infantry  of  the 
New  Jersey  Army  National  Guard.  The  OpFor  was  an 
insurgent- style  enemy  force. 

Each  day  was  characterized  by  a  unique  mission 
designed  to  test  the  operational  functionalities  of  the 
C4ISR  equipment.  Each  daily  run  consisted  of  a  pre¬ 
mission  briefing,  execution  of  the  mission,  a  mission 
debrief  and  a  mission  planning  session  for  the  next  day’s 
mission. 

The  pre-mission  brief  provided  the  Blue  Force  with 
their  mission  objective,  any  intelligence  known  about  the 
OpFor  and  instruction  on  how  the  Blue  Force  would 
advance  on  their  objective.  During  execution  of  the 
mission  C4ISR  assets  were  deployed  in  such  a  way  that 
the  quality  and  fidelity  of  information  provided  to/from 
the  Blue  Force  was  varied  to  study  the  effects  on  the 
network  and  the  commanders’  ability  to  attain  situational 
awareness  and  make  effective  decisions. 

After  each  mission  a  debriefing,  referred  to  as  the 
daily  hot  wash,  was  conducted.  The  debrief  was  a  recap 
of  the  day’s  events  and  observations  made  in  the 
command  post  and  the  field  that  were  relevant  to  the 
performance  of  the  C4ISR  equipment  and  the  soldiers’ 
use  of  that  equipment. 

Each  day  ended  with  a  mission  planning  session  for 
the  next  day’s  mission.  During  these  sessions 
commanders,  platoon  leaders  and  platoon  sergeants  would 
discussion  the  mission’s  objectives,  how  they  would 
deploy  their  C4ISR  equipment  and  what  tactics  they 
would  use  to  accomplish  their  objectives. 

5.  ROLE  OF  THE  ROVR 

The  ROVR™  supported  the  experiment  as  an 
advanced  visualization  tool  for  conducting  mission 
planning,  situational  awareness  (SA)  and  after  action 
reviews  (AARs).  Although  ROVR™  team  members 
evaluated  both  the  system’s  ability  to  provide  accurate  SA 
during  mission  execution  as  well  as  properly  replayed 
data  for  AARs,  the  only  evaluation  performed  on  the 
ROVR™  in  terms  of  its  impact  on  mission  execution  was 
during  mission  planning. 
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5.1  Mission  Planning  Session 

Due  to  its  ability  to  display  of  high-fidelity  terrain 
models,  the  ROVR™  system  was  utilized  to  perform 
scenario  simulation  for  a  mission  that  was  executed  at  a 
Military  Operations  in  Urban  Terrain  (MOUT)  site.  The 
objective  of  this  planning  session  was  to  evaluate  the 
impact  of  using  the  ROVR™  system  on  planning  and 
execution  of  the  mission. 

The  planning  session  was  attended  by  the  Platoon 
Leader,  Platoon  Sergeant  and  the  Robotics  NCO.  The 
goals  of  the  session  included  determining  the  best 
locations  for  placement  of  UGS,  identifying  suspected 
enemy  strongholds  and  safe  houses,  determining  the  best 
routes  of  ingress  and  identifying  a  possible  foothold  in  the 
site  for  the  FFW  Infantry  Platoon.  Figure  4  shows  the 
mission  planning  session. 


Figure  4.  Mission  planning  session 


To  expedite  the  session,  experienced  ROVR™ 
operators  performed  navigation  tasks  with  the  6DOF 
wand  to  alleviate  the  need  for  training.  Following  the 
planning  session,  soldiers  were  given  the  opportunity  to 
use  the  wand  so  that  feedback  on  its  use  could  be  attained. 
Figure  5  shows  a  soldier  using  the  6DOF  wand. 


The  first  task  for  the  soldiers  was  to  get  a  general 
orientation  of  the  layout  of  the  MOUT  site  and  determine 
the  best  locations  to  place  their  UGS.  The  ROVR™ 


operator  flew  them  over  the  site  to  get  an  aerial  view,  see 
Figure  6. 


Figure  6.  MOUT  aerial  view 


From  the  aerial  view  the  soldiers  could  identify 
routes  leading  into  and  out  of  the  MOUT  site.  As  the 
system  allows  the  operator  to  freely  navigate  within  the 
environment,  the  soldiers  used  a  birds-eye  view  of  the 
MOUT  site  street  layout  for  improved  orientation.  Based 
on  intelligence  reports  soldiers  could  predict  where 
enemy  forces  would  enter  the  site  and  were  able  to 
determine  approximate  locations  for  their  UGS. 

In  order  to  provide  the  soldiers  with  a  closer  look  at 
the  terrain  for  UGS  placement,  the  operator  navigated 
down  to  a  dismount  or  first-person  perspective  street  level 
for  a  closer  inspection.  Once  the  inspection  task  was 
completed  the  soldiers  walked  through  the  streets  of  the 
site  virtually  to  determine  best  routes,  identify  specific 
buildings,  and  identify  any  obstacles  that  may  impede 
their  movement.  Figure  7  shows  a  street  level  view  of  the 
MOUT  site. 


Figure  7.  MOUT  street  level 


The  soldiers  then  walked  around  the  suspected 
enemy  safe  houses  and  strongholds  to  determine  entrance 
ways,  windows  and  leam  the  general  characteristics  of 
each  building  from  all  four  sides.  They  then  identified  a 
potential  foothold  for  the  FFW  Platoon.  They  were  able 
to  leam  the  general  layout  of  that  location  and  identified 
routes  of  ingress  and  egress  to/from  the  building. 
Throughout  the  mission  planning  session  the  Robotics 
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NCO  took  notes  and  drew  diagrams  of  the  MOUT  site. 
After  the  planning  session  the  three  leaders  returned  to 
their  troops  to  train  them  using  their  notes  and  maps  of 
the  MOUT  site. 

5.2  Soldiers  Response  to  ROVR™ 

“Each  of  the  three  leaders  stated  that  this  planning 
session  was  a  great  benefit  to  their  planning.  The  Platoon 
Leader  estimated  that  the  technology  improved  his 
planning  capabilities  by  75%.  The  Platoon  Sergeant 
stated  that  the  technology  is  outstanding  if  it  is  accurate 
and  up  to  date.  He  thought  the  colors  and  shapes  of  the 
buildings  and  the  layout  of  the  streets  was  very  helpful. 
His  estimate  of  how  this  improved  their  planning  was 
70%.  The  Robotics  NCO  estimated  that  the  ROVR™ 
planning  was  80%  better  than  using  a  sand  table”  (E. 
Bowman,  Personal  Communication,  4  Aug  2006).  The 
soldiers  did  note  some  drawbacks  to  the  session.  The 
Platoon  Leader  said  that  he  initially  got  a  headache  from 
some  of  the  extreme  movements  through  the  scene. 
ROVR™  team  members  noted  that  this  is  typical  for 
some  new  users  due  to  the  stereoscopic  3D  view  and 
navigating  through  the  scenes  too  quickly.  Other 
comments  were  suggestions  for  functions  that  the  soldiers 
would  like  to  have  built  into  the  system.  These  functions 
include  the  ability  to  draw  on  or  markup  a  scene,  the 
ability  to  measure  distance  between  two  points,  and  the 
ability  to  print  out  a  marked-up  map.  The  ability  to 
markup  a  scene  is  available  in  the  Two-Handed  Interface 
(discussed  below),  however,  that  software  release  was  not 
available  at  the  time  of  the  mission  planning  session. 

5.3  Mission  Execution 

“On  the  morning  of  the  MOUT  mission,  the  Platoon 
Sergeant  was  confident  in  the  locations  of  the  buildings  of 
interest  in  the  area  and  knew  where  his  soldiers  and 
unmanned  assets  were  to  be  placed.  This  technology 
provided  an  excellent  tool  for  the  Soldiers  to  become 
familiar  with  the  layout  of  the  MOUT  site”  (E.  Bowman, 
Personal  Communication,  4  Aug  2006). 

When  the  Recon  platoon  arrived  at  the  MOUT  site 
to  execute  their  mission  they  did  so  with  very  little 
deviation  from  their  plan,  “the  PacBot  had  to  be  manually 
placed  on  a  rooftop  because  there  were  no  stairs  in  the 
building.  Otherwise,  Soldiers  and  systems  were  emplaced 
according  to  plan.” 

“The  Soldiers’  consensus  on  the  ability  of  the 
ROVR™  technology  to  improve  planning  and  execution 
was  very  favorable.  They  would  like  to  use  it  more  in  the 
future.  The  visualization  technology  is  realistic  and  it  is 
helpful  to  have  different  views  (ground  and  air)”  (E. 
Bowman,  Personal  Communication,  4  Aug  2006). 


6.  C2  TWO-HANDED  INTERFACE 

The  C2  Two-Handed  Interface  (C2THI)  application 
is  focused  on  extending  the  3D  C2  environment  by 
adapting  distributed  immersive  collaboration  capabilities. 
This  will  enable  remote  commanders  to  perform  advanced 
mission  planning/rehearsal  as  if  they  were  all  sitting 
together  in  the  same  room. 

With  the  SpaceGrip®  interaction  device,  hand-held 
devices  that  communicate  hand  position  and  orientation, 
users  of  the  system  navigate  through  scenes  using  the 
natural  movements  of  their  hands  and  arms.  Figure  8 
shows  how  well  SpaceGrips®  fit  comfortably  in  each 
hand. 


Figure  8.  C2THI  SpaceGrips® 

When  collaborating  together,  remote  users  show  up 
as  robot-like  avatars  in  the  other  users’  scene.  Each  user 
can  mark  up  a  scene  with  tactically  relevant  information 
using  the  3D  white  board.  The  white  board  is  then  shared 
with  each  participant.  Using  the  SpaceGrips® 
participants  can  even  gesture  to  each  other.  Figure  9 
shows  a  marked  up  screen,  i.e.  the  locations  of  Blue  and 
Red  controlled  areas  and  a  remote  user’s  avatar. 


Figure  9.  C2THI  collaborative  session 
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7.  THE  WAY  AHEAD 

Leveraging  the  successes  of  the  2006  experiment, 
the  CAVE  team  will  continue  to  develop  its  hardware  and 
software  to  meet  the  needs  of  the  soldiers  for  whom  it  is 
intended.  Feedback  from  soldiers  and  analysts  will  guide 
the  coming  year’s  development. 

The  CAVE  team  will  address  the  feedback  from  the 
soldiers  who  used  the  ROVR™,  such  as  being  able  to 
export  marked  up  scenes  to  other  Army  Battle  Command 
Systems  as  Overlay  messages.  We  will  add  capabilities 
such  as  overlaying  grid  lines  on  the  terrain  and  the  ability 
to  “hook”  a  track  and  receive  detailed  information  about 
an  entity. 

The  CAVE  team  will  also  address  hardware  issues 
such  as  the  “footprint”  of  its  current  systems.  Fakespace 
is  currently  developing  increasingly  portable  systems  that 
may  be  more  suitable  for  placement  within  smaller 
operations  centers  without  losing  the  immersive  effects  of 
the  current  ROVR™. 

The  CAVE  team  will  continue  to  improve  the  visual 
fidelity  of  the  modeled  3D  Fort  Dix  environment  through 
the  introduction  of  more  ranges  and  extensive  foliage.  A 
more  automated  method  of  3D  data  acquisition  and  model 
creation  will  be  investigated  and  applied. 

Finally,  the  CAVE  team  will  seek  to  perform  a  more 
extensive  field  evaluation  of  its  systems  at  the  C4ISR 
OTM  2007  Experiment  including  the  areas  of  situational 
awareness  and  after  action  reviews. 


Bowman,  Personal  Communication,  4  Aug  2006).  The 
ROVR™  planning  session  resulted  in  the  soldiers 
spending  less  time  in  harms  way  with  increased 
effectiveness. 
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8.  CONCLUSION 

Immersive  display  technologies  have  the  potential  to 
assist  commanders  in  receiving  and  understanding  large 
amounts  of  tactical  data.  Initial  testing  and  results  from 
the  C4ISR  OTM  2006  Experiment  indicate  the  soldiers 
liked  its  capabilities,  and  thought  the  ROVR™  was  a  vast 
improvement  over  current  mission  planning  methods. 
During  the  planning  session  leaders  used  aerial  views  to 
provide  a  “broad  view  to  plan  initial  sensor/soldier 
placement.  Soldiers  used  the  street-level  views  to  walk 
around  buildings  and  finalize  their  plans”  (E.  Bowman, 
Personal  Communication,  4  Aug  2006). 

Observations  during  the  execution  of  the  mission 
indicate  the  ROVR™  solution  was  instrumental  in 
assisting  soldiers  to  move  through  the  area  of  interest  and 
emplacement  of  assets.  “Benefits  for  execution  were 
greater  knowledge  of  the  objective  area,  increased 
confidence,  shortened  time  for  emplacement  of 
technologies  by  soldiers,  and  rapid  identification  of  key 
buildings  based  on  color  shape  and  location”  (E. 
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